iT邦幫忙

2021 iThome 鐵人賽
DAY 16
0
AI & Data

AI平台初學者工作坊: 從training、tracking到serving系列 第 16 篇

細看seldon core所部署出來的POD在做什麼

13th鐵人賽
1665 瀏覽

  • 分享至 

  • xImage
    •  

在本篇, 我們來看一下使用seldon完成部署之後, 在k8s上會產生哪些資源

建立在k8s上的seldon資料

  • POD
    有一個POD, 在這個POD裡面包含2個container.
NAME                                                  READY   STATUS    RESTARTS   AGE
pod/xgboost-default-0-classifier-9688db8bf-9x2bf      2/2     Running   0          16h
  • Service
    有二個service
NAME                                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
service/xgboost-default              ClusterIP   10.99.29.1      <none>        8000/TCP,5001/TCP   16h
service/xgboost-default-classifier   ClusterIP   10.110.241.88   <none>        9000/TCP,9500/TCP   16h
  • Deployment
    有一個demployment.
NAME                                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/xgboost-default-0-classifier      1/1     1            1           16h
  • Replicaset
NAME                                                        DESIRED   CURRENT   READY   AGE
replicaset.apps/xgboost-default-0-classifier-9688db8bf      1         1         1       16h

細看POD內容

在這些資源中, 我們來看一下xgboost-default-0-classifier-9688db8bf-9x2bf這個POD為我們做些什麼.

  • 查看POD內容
    使用kubectl describe指令, 我們可以知道這個POD包含一個initContainer(classifier-model-initializer)與2個container(classifier與seldon-container-engine)
kubectl describe pod xgboost-default-0-classifier-9688db8bf-9x2bf
  • 查看initContainer的log
    指令下如:
kubectl logs xgboost-default-0-classifier-9688db8bf-9x2bf classifier-model-initializer

[output]
可以看出, initContainer的主要功能是讀取model檔. 在我們的範例中使用xbgboost server, 因此必需將model的檔案命名為model.bst. 可以參考這裡的log判斷是否可以正確讀取model檔.

2021/09/07 13:35:24 NOTICE: Config file "/.rclone.conf" not found - using defaults
2021/09/07 13:35:24 DEBUG : rclone: Version "v1.55.1" starting with parameters ["rclone" "copy" "-vv" "/mnt/pvc/" "/mnt/models"]
2021/09/07 13:35:24 DEBUG : Creating backend with remote "/mnt/pvc/"
2021/09/07 13:35:24 DEBUG : Creating backend with remote "/mnt/models"
2021/09/07 13:35:25 DEBUG : Local file system at /mnt/models: Waiting for checks to finish
2021/09/07 13:35:25 DEBUG : Local file system at /mnt/models: Waiting for transfers to finish
2021/09/07 13:35:25 DEBUG : model.bst: MD5 = 0c2a00b4f880120021c164d7c238edc0 OK
2021/09/07 13:35:25 INFO  : model.bst: Copied (new)
2021/09/07 13:35:25 DEBUG : model.bst.xgbjson: MD5 = 53962cbec628e763c7d9c33534ba860c OK
2021/09/07 13:35:25 INFO  : model.bst.xgbjson: Copied (new)
2021/09/07 13:35:25 INFO  :
Transferred:        1.944M / 1.944 MBytes, 100%, 24.475 MBytes/s, ETA 0s
Transferred:            5 / 5, 100%
Elapsed time:         0.3s

2021/09/07 13:35:25 DEBUG : 2 go routines active
  • 查看 classifier container
    這裡的container名稱是 classifier, 是我們在yaml檔所給定的名稱.
    指令如下:
kubectl logs xgboost-default-0-classifier-9688db8bf-9x2bf classifier

[output]
在output中可以看出, classifier這個container的主要功能是提供endpoint, port是9000.
當你存取endpoint如果發生失敗時, 可以來看這裡的log內容, 查看每次打endpoint時在server端發生了什麼事.

starting microservice
2021-09-07 13:35:32,077 - seldon_core.microservice:main:206 - INFO:  Starting microservice.py:main
2021-09-07 13:35:32,077 - seldon_core.microservice:main:207 - INFO:  Seldon Core version: 1.10.0
2021-09-07 13:35:32,083 - seldon_core.microservice:main:362 - INFO:  Parse JAEGER_EXTRA_TAGS []
2021-09-07 13:35:32,084 - seldon_core.microservice:load_annotations:158 - INFO:  Found annotation cni.projectcalico.org/containerID:41bc106f063b21c2b1d2cc9a2b65545663881f02e7651b1c986e0a56c6446031
2021-09-07 13:35:32,084 - seldon_core.microservice:load_annotations:158 - INFO:  Found annotation cni.projectcalico.org/podIP:10.244.79.109/32
2021-09-07 13:35:32,085 - seldon_core.microservice:load_annotations:158 - INFO:  Found annotation cni.projectcalico.org/podIPs:10.244.79.109/32
2021-09-07 13:35:32,085 - seldon_core.microservice:load_annotations:158 - INFO:  Found annotation kubernetes.io/config.seen:2021-09-07T21:35:18.889938774+08:00
2021-09-07 13:35:32,085 - seldon_core.microservice:load_annotations:158 - INFO:  Found annotation kubernetes.io/config.source:api
2021-09-07 13:35:32,085 - seldon_core.microservice:load_annotations:158 - INFO:  Found annotation prometheus.io/path:/prometheus
2021-09-07 13:35:32,086 - seldon_core.microservice:load_annotations:158 - INFO:  Found annotation prometheus.io/scrape:true
2021-09-07 13:35:32,086 - seldon_core.microservice:main:365 - INFO:  Annotations: {'cni.projectcalico.org/containerID': '41bc106f063b21c2b1d2cc9a2b65545663881f02e7651b1c986e0a56c6446031', 'cni.projectcalico.org/podIP': '10.244.79.109/32', 'cni.projectcalico.org/podIPs': '10.244.79.109/32', 'kubernetes.io/config.seen': '2021-09-07T21:35:18.889938774+08:00', 'kubernetes.io/config.source': 'api', 'prometheus.io/path': '/prometheus', 'prometheus.io/scrape': 'true'}
2021-09-07 13:35:32,086 - seldon_core.microservice:main:369 - INFO:  Importing XGBoostServer
2021-09-07 13:35:32,719 - seldon_core.microservice:main:448 - INFO:  REST gunicorn microservice running on port 9000
2021-09-07 13:35:32,723 - seldon_core.microservice:main:542 - INFO:  REST metrics microservice running on port 6000
2021-09-07 13:35:32,723 - seldon_core.microservice:main:552 - INFO:  Starting servers
2021-09-07 13:35:32,742 - seldon_core.microservice:grpc_prediction_server:505 - INFO:  GRPC Server Binding to '%s' 0.0.0.0:9500 with 1 processes
2021-09-07 13:35:32,753 - seldon_core.wrapper:_set_flask_app_configs:224 - INFO:  App Config:  <Config {'ENV': 'production', 'DEBUG': False, 'TESTING': False, 'PROPAGATE_EXCEPTIONS': None, 'PRESERVE_CONTEXT_ON_EXCEPTION': None, 'SECRET_KEY': None, 'PERMANENT_SESSION_LIFETIME': datetime.timedelta(days=31), 'USE_X_SENDFILE': False, 'SERVER_NAME': None, 'APPLICATION_ROOT': '/', 'SESSION_COOKIE_NAME': 'session', 'SESSION_COOKIE_DOMAIN': None, 'SESSION_COOKIE_PATH': None, 'SESSION_COOKIE_HTTPONLY': True, 'SESSION_COOKIE_SECURE': False, 'SESSION_COOKIE_SAMESITE': None, 'SESSION_REFRESH_EACH_REQUEST': True, 'MAX_CONTENT_LENGTH': None, 'SEND_FILE_MAX_AGE_DEFAULT': datetime.timedelta(seconds=43200), 'TRAP_BAD_REQUEST_ERRORS': None, 'TRAP_HTTP_EXCEPTIONS': False, 'EXPLAIN_TEMPLATE_LOADING': False, 'PREFERRED_URL_SCHEME': 'http', 'JSON_AS_ASCII': True, 'JSON_SORT_KEYS': True, 'JSONIFY_PRETTYPRINT_REGULAR': False, 'JSONIFY_MIMETYPE': 'application/json', 'TEMPLATES_AUTO_RELOAD': None, 'MAX_COOKIE_SIZE': 4093}>
2021-09-07 13:35:32,777 - seldon_core.wrapper:_set_flask_app_configs:224 - INFO:  App Config:  <Config {'ENV': 'production', 'DEBUG': False, 'TESTING': False, 'PROPAGATE_EXCEPTIONS': None, 'PRESERVE_CONTEXT_ON_EXCEPTION': None, 'SECRET_KEY': None, 'PERMANENT_SESSION_LIFETIME': datetime.timedelta(days=31), 'USE_X_SENDFILE': False, 'SERVER_NAME': None, 'APPLICATION_ROOT': '/', 'SESSION_COOKIE_NAME': 'session', 'SESSION_COOKIE_DOMAIN': None, 'SESSION_COOKIE_PATH': None, 'SESSION_COOKIE_HTTPONLY': True, 'SESSION_COOKIE_SECURE': False, 'SESSION_COOKIE_SAMESITE': None, 'SESSION_REFRESH_EACH_REQUEST': True, 'MAX_CONTENT_LENGTH': None, 'SEND_FILE_MAX_AGE_DEFAULT': datetime.timedelta(seconds=43200), 'TRAP_BAD_REQUEST_ERRORS': None, 'TRAP_HTTP_EXCEPTIONS': False, 'EXPLAIN_TEMPLATE_LOADING': False, 'PREFERRED_URL_SCHEME': 'http', 'JSON_AS_ASCII': True, 'JSON_SORT_KEYS': True, 'JSONIFY_PRETTYPRINT_REGULAR': False, 'JSONIFY_MIMETYPE': 'application/json', 'TEMPLATES_AUTO_RELOAD': None, 'MAX_COOKIE_SIZE': 4093}>
2021-09-07 13:35:32,791 - seldon_core.microservice:_run_grpc_server:460 - INFO:  Starting new GRPC server with 1.
[2021-09-07 13:35:32 +0000] [36] [INFO] Starting gunicorn 20.1.0
[2021-09-07 13:35:32 +0000] [36] [INFO] Listening at: http://0.0.0.0:6000 (36)
[2021-09-07 13:35:32 +0000] [36] [INFO] Using worker: sync
2021-09-07 13:35:32,926 - root:download:31 - INFO:  Copying contents of /mnt/models to local
[2021-09-07 13:35:32 +0000] [46] [INFO] Booting worker with pid: 46
[2021-09-07 13:35:33 +0000] [7] [INFO] Starting gunicorn 20.1.0
[2021-09-07 13:35:33 +0000] [7] [INFO] Listening at: http://0.0.0.0:9000 (7)
[2021-09-07 13:35:33 +0000] [7] [INFO] Using worker: sync
[2021-09-07 13:35:33 +0000] [48] [INFO] Booting worker with pid: 48
2021-09-07 13:35:33,152 - seldon_core.gunicorn_utils:load:103 - INFO:  Tracing not active
2021-09-07 13:35:33,153 - root:download:31 - INFO:  Copying contents of /mnt/models to local

  • 查看 seldon-container-engine
    指令如下:

    kubectl logs xgboost-default-0-classifier-9688db8bf-9x2bf seldon-container-engine

    [output]
    這是seldon core所以提供的功能(SeldonRestApi). 這個container應該不用花太多時間去看它, 只是確定這個container狀態是running就好.

    {"level":"info","ts":1631021729.002178,"logger":"entrypoint","msg":"Hostname unset will use localhost"}
{"level":"info","ts":1631021729.092143,"logger":"entrypoint","msg":"Starting","worker":1}
{"level":"info","ts":1631021729.0923526,"logger":"entrypoint","msg":"Starting","worker":2}
{"level":"info","ts":1631021729.0924156,"logger":"entrypoint","msg":"Starting","worker":3}
{"level":"info","ts":1631021729.09253,"logger":"entrypoint","msg":"Starting","worker":4}
{"level":"info","ts":1631021729.0925732,"logger":"entrypoint","msg":"Starting","worker":5}
{"level":"info","ts":1631021729.095814,"logger":"entrypoint","msg":"Running http server ","port":8000}
{"level":"info","ts":1631021729.0958958,"logger":"entrypoint","msg":"Creating non-TLS listener","port":8000}
{"level":"info","ts":1631021729.0968246,"logger":"entrypoint","msg":"Running grpc server ","port":5001}
{"level":"info","ts":1631021729.0968761,"logger":"entrypoint","msg":"Creating non-TLS listener","port":5001}
{"level":"info","ts":1631021729.096972,"logger":"entrypoint","msg":"Setting max message size ","size":2147483647}
{"level":"info","ts":1631021729.100623,"logger":"SeldonRestApi","msg":"Listening","Address":"0.0.0.0:8000"}

細看Service內容

前面提到有2個service

NAME                                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
service/xgboost-default              ClusterIP   10.99.29.1      <none>        8000/TCP,5001/TCP   16h
service/xgboost-default-classifier   ClusterIP   10.110.241.88   <none>        9000/TCP,9500/TCP   16h

我們是透過xgboost-default-classifier這個service取得推論的結果. 我們來查一下這兩個serivce是連到哪個pod

kubectl get ep

結果是

NAME                                            ENDPOINTS                                            AGE
cluster.local-nfs-subdir-external-provisioner   <none>                                               25d
kubernetes                                      172.23.180.10:6443                                   25d
xgboost-default                                 10.244.79.109:5001,10.244.79.109:8000                3d17h
xgboost-default-classifier                      10.244.79.109:9000,10.244.79.109:9500                3d17h

可以看出xgboost-default-classifier這個service的endpoint是指向xgboost-default-0-classifier-9688db8bf-9x2bf這個POD(10.244.79.109), 因此我們可以在前一篇進行推論時經由service最得推論結果

到這裡我們已經完成Seldon Core安裝, 將model部署在seldon之上, 並且提供endpoint可供存取.


上一篇
部署model(serving) on seldon(PVC)
下一篇
安裝MinIO並從notebook儲存model到MinIO
系列文
AI平台初學者工作坊: 從training、tracking到serving 共 30 篇
  1. 26
    口罩臉孔資料集的標註、資料前處理與資料擴增
  2. 27
    口罩臉孔資料集的訓練(Training)
  3. 28
    建立Endpoint執行二次開發
  4. 29
    在Edge上進行部署(Serving)
  5. 30
    結語: AI平台的六個ING
完整目錄
圖片
  直播研討會
圖片
{{ item.channelVendor }} {{ item.webinarstarted }} |
{{ formatDate(item.duration) }}
直播中

尚未有邦友留言

立即登入留言
iThome鐵人賽
參賽組數
1123 組
團體組數
52 組
累計文章數
23096 篇
完賽人數
656 人
iT+看影片追技術 看影片追技術 看更多
熱門tag 看更多
15th鐵人賽 13th鐵人賽 14th鐵人賽 12th鐵人賽 11th鐵人賽 鐵人賽 2019鐵人賽 2018鐵人賽 javascript 2017鐵人賽 python windows php c# windows server linux css 程式設計 react vue.js
熱門問題
熱門回答
熱門文章
IT邦幫忙